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© Packet switching system arranged for congestion control. 

©A method for controlling congestion in a packet 
switching network uses marked packets and a pack- 
et dropping algorithm to determine when to drop the 
marked packets wherever the network is congested 
at any point along the path being traversed by the 
"marked packets. 
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PACKET SWITCHING SYSTEM ARRANGED FOR CONGESTION CONTROL 



Technical Field 

This invention relates to a packet switching 
system arranged for controlling switch node and 
link congestion caused by customers transmitting 
information at excessive rates. 



Background of the Invention 

Packet communication involves a technique of 
disassembling information at the sending end of a 
switching network for insertion into separate bursts, 
or packets, of data and reassembling the same 
information from the data packets at the receiving 
end of the network. Communication according to 
this technique is especially useful in common car- 
rier or time-shared switching systems, since the 
communication path or circuit required for the 
packet transmissions is needed only while each 
packet is being forwarded through the network, and 
is, therefore, available to other users during inter- 
vening periods. 

Packet switching offers another attractive fea- 
ture. That is the flexibility of providing integrated 
information transport services for a wide range of 
applications, e.g., interactive data, bulk data, sig- 
naling, packetized voice, image, etc. Instead of 
designing specialized networks optimized for spe- 
cific forms of applications, many services can be 
simultaneously operated over the same connection 
to the network. All varieties of user information are 
converted into packets, and the network transports 
these packets between users. End users are not 
tied to fixed rat connections. Instead, network con- 
nections adapt to the particular needs of the end 
users. Furthermore, it is possible to create a uni- 
form user-network interface applicable to a broad 
range of services. Note that different applications 
may require different grades of service from the 
network. For example, packetized voice transmis- 
sion has very stringent delay requirements for de- 
livery of associated packets of an ongoing con- 
versation. 

Efficient utilization of network- resources can be 
attained by allowing packetized transmissions of a 
plurality of users on the same connection oh a 
time-shared basis. Thus the packets of one user 
are interspersed with the packets of other users. 

Elements of the resources or facilities which 
may be shared in such packet networks include 
transmission link bandwidth (defined as bytes/sec, 
a measure of link capacity), processor real time 
(i.e., time immediately available for processing 



packets), ports or links, and data or packet buffers. 
In large multinode networks, each node or packet 
switch accommodates many such ports or links 
that terminate paths which extend to users 1 termi- 

s nal equipments or to other nodes. Each node may 
include one or more processors for controlling the 
routing and processing of packets through the 
node. The node is customarily equipped with a 
large number of buffers for storing packets prior to 

70 such routing or while awaiting an output link. Each 
line between nodes or extending to users typically 
serves a plurality of concurrent calls between dif- 
ferent terminal equipments. Each packet passing 
through the network consumes a certain amount of 

is processor real time at each node, takes away a 
certain amount of link capacity (proportional to the 
packet size), and occupies buffers while being pro- 
cessed. There is a maximal number of packets per 
unit of time that a network can accommodate. This 

20 notion of "capacity" depends on all the aforemen- 
tioned resources provisioned within the network, as 
well as on the particular traffic mix generated by 
the users. 

One problem in a packet switching system 
25 arises when many users attempt to utilize the net- 
work at the same time. This results in the formation 
of many paths or circuits for routing the packets 
and the congestion of the communication facilities. 
Congestion of a facility is the occurrence of more 
30 work than can be handled by the facility in a 
specific period of time. It has been found that 
congestion tends to spread through the network if 
the congestion is uncontrolled. As a result, it is 
desirable to have a flow/congestion control mecha- 
35 nism for protecting the expected performance level 
for each service type (e.g., voice) from unpredict- 
able traffic overloads due to other service types. 
Protection from overload can be provided through 
the allocation of key network resources. In the 
40 event that a key resource is overloaded by traffic, it 
is desirable that the overall performance of the 
system should degrade as gracefully as possible. 
Controlling the utilization of the key resource may 
require different objectives under the overload con- 
45 dition than under a normal load condition. 

A principal area of packet congestion is in 
buffers, or queues, in each node, particularly where 
the buffers become unavailable to store incoming 
packets. Yet the buffer requirement is closely re- 
so lated to the utilization of processor real time and/or 
link bandwidth. When the processor real time is 
exhausting, or when the link bandwidth is not suffi- 
cient to handle the packet traffic, queues within the 
switching node will build up causing a long delay. 
Finally packet buffers will be exhausted, resulting 
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in the dropping of packets. A number of flow con- 
trol procedures, e.g., end-to-end windowing, have 
been developed and commercially exploited for 
controlling congestion. 

The known end-to-end windowing scheme for 
flow control is advantageous when network opera- 
tion is viewed strictly from the network periphery. 
Each machine can have many logical channels 
simultaneously established between itself and var- 
ious other machines. For each of these logical 
channels, a given machine is allowed to have W 
unacknowledged packets outstanding in the net- 
work. For example, a machine can initially transmit 
W packets into the network as fast as it desires; 
but, thereafter, it must wait until it has received an 
acknowledgement from the destination machine for 
at least one of those outstanding packets before it 
can transmit more packets. 

This scheme has several very desirable prop- 
erties. There is very little wasted link bandwidth 
caused by the flow-controiling mechanism because 
the number of bits in an acknowledgement can be 
made very small compared to the number of bits in 
the W packets to which it refers. There is an 
automatic throttling of transmission under a heavy 
load condition because the increased round-trip 
delay will slow down the acknowledgements and 
hence the traffic source. 

There also are disadvantages to the end-to-end 
window flow control. The windowing mechanism by 
itself is not robust enough. The mechanism relies 
upon the end user adhering to an agreed-upon 
window size. By unilaterally increasing its window 
size, an abusive user can get favorable perfor- 
mance while degrading the performance for other 
users. Even if all users obey their specified end-to- 
end window sizes, it is very difficult to determine 
suitable window sizes for various load conditions. In 
general, the window size W is chosen large enough 
to aJlow uninterrupted transmission when the net- 
work is lightly loaded; however, overload conditions 
may require an unacceptably large amount of buff- 
er storage within the packet switch. It is possible 
for end users to adaptively adjust their window 
sizes based on network congestion, but this by 
itself would not necessarily give a fair congestion 
control. 

Another drawback to relying upon the end-to- 
end windowing mechanism is that not all agreed- 
upon user applications are subject to window- 
based end-to-end control. For example,- the Un- 
numbered Information (Ul) transfer embedded in 
some window-based protocols, e.g.. LAPD. allows 
the end users to send packets without any win- 
dowing limitation. Other examples are packetized 
voice or packetized voice-band data applications 
where an end-to-end window is not applicable. 

It has been proposed that in an integrated 



voice/data packet network the proper way to control 
flow/congestion is by allocating bandwidth to con- 
nections and by making new connections only 
when the needed bandwidth is available. 
5 This means that the network must provide a 

mechanism for users to select their bandwidth 
needs and indicate the burstiness of their transmis- 
sions. Thereafter, the network must enforce those 
parameters with respect to the respective users. 
ro A key part of bandwidth allocation is the 

mechanism used to select and specify the needed 
bandwidth and to limit users to their selections. 
Perhaps the simplest approach is the so-called 
"leaky bucket" method. A count in a counter, asso- 

75 dated with each user terminal transmitting on a 
.connection, is incremented whenever the user 
sends a packet and is decremented periodically. 
The user selects the rate at which the count is 
decremented (this determines the average band- 

20 width) and the value of a threshold (a number 
indicating burstiness). If the count exceeds the 
threshold upon being incremented, the network dis- 
cards that packet. 

There are problems with this rt leaky bucket" 

25 bandwidth allocation proposal. A major problem is 
the fact that the control is open-loop in nature. A 
user's packets will be dropped once the threshold 
is exceeded even when the network could have 
handled the packets. Precious network resources 

30 would be wasted. The unnecessary throttling of that 
user's data may sustain the information transfer 
over a lengthy period, contributing to network con- 
gestion at a later time. Another problem is that the 
limiting network resource may be processor real 

35 time rather than link bandwidth. 



Summary of the Invention 

40 These and other problems are resolved by a 

novel method and apparatus for controlling conges- 
tion in a packet switching network. The method 
uses a packet dropping algorithm to determine 
when to drop marked packets being transmitted 

45 through the network. Market packets are dropped 
wherever the network is congested at any point 
along the path being traversed by the data packets. 

A copending patent application defining the 
method utilized for monitoring and marking packets 

so has been filed concurrently herewith in the names 
of A. E. Eckberg, Jr.-D. T. Lucan-D. M Lucantoni-T. 
J. Schonfield Case 1-1-1-1. 
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Brief Description of the Drawings 

The invention may be better understood by 
reading the following detailed description with ref- 
erence to the drawings wherein 

FIG. 1 illustrates a packet switching network 
arranged for interconnecting voice/data/video termi- 
nals and controlling congestion within the network; 

FIG. 2 is a block diagram of circuitry ar- 
ranged for monitoring the rate of transmission of a 
customer and for marking that customer's packets; 

FIG. 3 shows a flow chart of an algorithm for 
monitoring the rate of transmission of a customer 
and for marking the customer's packets when the 
rate of transmission is excessive; 

FIG. 4 presents a flow chart of an algorithm 
for periodically adjusting a counter used in monitor- 
ing the rate of transmission of a customer and for 
determining a dynamic threshold relating to a cus- 
tomers selected rate of transmission; 

FIG. 5 is a graphic illustration of the actions 
of the algorithms shown in FIGS. 3 and 4 for a 
sequence of packets from a customer; 

FIG. 6 is a block diagram of circuitry ar- 
ranged for dropping marked packets within the 
network; 

FIG. 7 shows a flow chart of an algorithm for 
dropping marked packets which encounter conges- 
tion within the network; 

FIG. 8 presents a flow chart of another al- 
gorithm for monitoring the rate of transmission of a 
customer and for marking the customer's packets 
when the rate of transmission is excessive; and 

FIG. 9 is a graphic illustration of the actions 
of the algorithms shown in FIGS. 4 and 8 for a 
sequence of packets from a customer. 



Detailed Description 

Referring now to FIG. 1, there is shown a 
packet switching network 20 that is arranged for 
establishing virtual circuit connections between pair 
of terminal equipments. Terminals 21 and 22 trans- 
mit packets of data through customer connection 
lines 25 and 26 to a packet multiplexer 28. Other 
terminals, not shown in FIG. 1 but indicated by a 
series of dots, also can transmit packets of data 
into the multiplexer 28. Although the terminals 21 
and 22 are shown as computer terminals, they also 
may be digitized voice, video, or other data termi- 
nals. A resulting output stream of packets, inter- 
spersed with one another, are transmitted from the 
multiplexer 28 over an access line 29 to an access 
node 30 in the packet switching network 20. An- 
other access line 33, and other access lines repre- 
sented by a series of dots, also transmit streams of 
data packets into the access node 30. Some of 
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these access lines originate at a multiplexer and 
, others originate at a high speed terminal equip- 
ment. 

Although a typical packet switching network 
s may be a very complex network of switch nodes 
and links, only five switch nodes 30, 40, 50. 60 and 
70 are shown in FIG. 1 to illustrate an arrangement 
of the invention. 

In FIG. 1 only node 30 is arranged as an 
yo access node for receiving packets for customers' 
terminal equipments. Any or all of other nodes 40, 
50, 60 or 70 may also be access nodes in an 
operating system, but are not shown as access 
nodes in the network 20 merely to simplify the 
is drawing. 

Node 60 is shown as an egress node in FIG. 1. 
The other nodes also may be egress nodes but are 
not shown as such in FIG. 1 to simplify the draw- 
ing. From the egress node 60, streams of packets 
20 are transmitted over egress lines 61 and 62 and 
others, represented by a series of dots, to demul- 
tiplexers or customers' equipments. For purposes 
of simplification of FIG. 1, only a single demul- 
tiplexer 63 is shown. The stream of data packets 
25 transmitted over the egress line 61 is separated 
within the demultiplexer 63 according to customer 
identity so that customer specific packets are 
passed over customer connection lines 65 and 66, 
respectively, to customer terminals 67 and 68. Oth- 
30 er customer lines and terminals also are supplied 
with streams of packets from the demultiplexer 63. 
Those other customer lines and terminals are re- 
presented in FIG. 1 by a series of dots. 

For purposes of illustrating the operation of the 
35 data switching network 20, an exemplary virtual 
connection is shown by a heavily weighed path line 
linking the terminal equipment 21 to the terminal 
equipment 67. Although typical transmission is two- 
way over such a virtual connection, only one-way 
40 transmission from the terminal equipment 21 
through the network 20 to the terminal equipment 
67 isshown in FIG. 1. This virtual circuit connection 
is established from the multiplexer 28 and access 
line 29 through the access node 30, switch nodes 
45 40 and 50, links 72, 74 and 76. egress node 60, 
and egress link 61 to the demultiplexer 63. 

The network 20 is arranged for congestion con- > 
trol. Links and switches are provisioned in quan- 
tities that permit unimpeded transmission of all 
so packets up to a limit. Congestion, which potentially * 
might occur at any point within the network 20, can 
impede the progress of a growing number of pack- 
ets if the congestion continues for an extended 
period of time. As a result, the congestion can 
55 spread throughout the network and disable the 
network from effective operation. 

The advantageous congestion control scheme, 
herein presented, is directed toward monitoring and 
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marking selected customer data packets and elimi- 
nating or dropping from further transmission 
through the network marked data packets whenever 
and wherever they encounter a congestion con- 
dition. This control scheme is implemented by al- 
gorithms which affect individual data packets at 
various times and points within the network as a 
continuing stream of packets progresses through 
the virtual connection. Each customer or end user 
may establish multiple virtual connections to dif- 
ferent customers of the network. The monitoring 
and marking scheme can be implemented on a per 
virtual circuit basis, on a per group of virtual cir- 
cuits basis, or on a per customer basis. To simplify 
further explanation, we assume there is only one 
virtual circuit per customer, so the terms customer 
and virtual circuit will be synonymous hereinafter. 

A first algorithm is for monitoring the bandwidth 
of a customer and for marking that customer's 
packets when that customer's subscribed band- 
width is exceeded. In this context, bandwidth may 
be defined as a two-dimensional quantity in the 
units (bytes/sec, packets/sec) to distinguish be- 
tween whether link bandwidth or processor real 
time is the limiting resource. The subscribed band- 
width is described in terms of an average rate (a 
throughout in bytes/sec that is guaranteed to be 
achievable by the user with packets of a specified 
size) and a burstiness factor -(where a measure of 
burstiness is, for example, the peak to mean ratio 
of the transmission rate as well as the duration of 
the peak transmissions). This first algorithm is used 
at the receive side of the access node 30. Each of 
the packets received from the access line 29 in- 
cludes information in a header for identifying the 
virtual circuit connection to which the packet be- 
longs. Accordingly the various packets are stored 
in registers and are identified with specific virtual 
circuits. 

Information from the headers of packets, being 
transmitted by the terminal equipment 21 and be- 
ing received by the access node 30, is applied to a 
bandwidth monitoring and packet marking circuit 
80 in FIG. 1. 

Referring now to FIG. 2. there is a block dia- 
gram of the circuit 80 which performs the proposed 
bandwidth monitoring the packet marking functions 
on a per customer basis. Only information from 
packets, identified as originating from the terminal 
21 of FIG. 1 and transmitted through the heavily 
weighted line linking to the terminal 67, is mon- 
itored by the algorithm. The circuit 80 is time 
shared for performing the same functions with re- 
spect to other virtual connections, but all of the 
monitoring and marking is done separately with 
respect to individual virtual connections. 

The monitoring is accomplished by an algo- 
rithm which determines whether or not the individ- 
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ual customer at terminal 21 is transmitting at an 
excessive rate (a rate greater than the subscribed 
rate) over the virtual circuit extending to the termi- 
nal 67. 

5 When the illustrated virtual connection is set 

up, the customer terminal equipment 21 and the 
network 20 negotiate for a desired bandwidth al- 
location relating to the virtual connection. The ban- 
dwidth allocation will be called the selected, or 

w subscribed, transmission rate. Information transmis- 
sions which exceed, or are greater than, the sub- 
scribed transmission rate are referred to as exces- 
sive rates. 

A processor in the access node 30 translates 
75 the subscribed transmission rate allocation into the 
long-term threshold M, a short-term threshold S 
and a decrement constant c. The long-term thresh- 
old M is chosen to accommodate the largest burst 
size allowed by the subscribed transmission rate. 

20 and the short-term threshold S is determined by 
the maximum instantaneous rate allowed. The de- 
crement constant c relates to the guaranteed aver- 
age throughput of unmarked packets. These are 
initializing parameters that are applied to a logic 

25 circuit 81 when the virtual connection is estab- 
lished, as shown in FIG. 2, for updating values of 
COUNT and THRESH at the end of an interval. 
They are used subsequently in the bandwidth mon- 
itoring and packet marking circuit 80. Also initially 

30 the value of COUNT in an accumulator 82 is set to 
zero and an active threshold value THRESH is set 
equal to S in a register 83. Further, in initialization, 
a parameter k, which is a weighting factor related 
to the number of packets per interval, is applied to 

35 a logic circuit 84 which produces a signal for 
marking a packet being transmitted at an excessive 
transmission rate. 

During each interval, as shown in FIGS. 2 and 
5. the number of bytes contained in each arriving 

40 packet is input by way of a lead 85 to the register 
storing BYTES to be applied to the logic circuit 84. 
Circuit 84 decides whether or not the specific pack- 
et should be marked as being transmitted at an 
excessive transmission rate. If the packet is to be 

45 marked, a marking signal is generated on the lead 
88 for inserting a marking signal into the packet 
header for identifying the packet as one being 
transmitted at an excessive transmission rate. If the 
packet is within the limits of the subscribed trans- 

so mission rate, no marking signal is generated or 
inserted into the header of the packet. 

There are three alternative packet marking al- 
gorithms to be described herein by way of illustra- 
tions. Others can be used as well. 

55 
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Parameters Used in the Algorithms (A) and ffi- 

I interval between successive decrements to 
the value of COUNT in the accumulator; this is a 
fixed interval for each virtual circuit being mon- 
itored and may differ among the virtual circuits; a 
typical value for I could be in the 10-500 msec 
range. 

k -a parameter by which the value of COUNT 
in the accumulator is to be incremented for each 
packet sent from the customer's terminal in addi- 
tion to .the value of BYTES for the packet; the 
parameter k represented a byte penalty per packet 
in guaranteed throughput that provides network 
protection from excessive minimal-sized packets 
that might otherwise stress real time resources; a 
typical value for the parameter k is a number 
between 0 and 1000; a value of this parameter k 
equal to zero would be used when processor real 
time is not a concern. 

c -a decrement constant relating to the cus- 
tomer's selected throughput of bytes per interval 
which will avoid packets being marked for possibly 
being dropped; an amount by which the value of 
COUNT in the accumulator is to be decremented 
during each interval; the number of a customer's 
bytes per interval (subscribed transmission rate) 
that will guarantee all transmitted packets are un- 
marked. 

S -a hort-term or instantaneous threshold on 
throughput during each interval which, if exceeded, 
will cause packets to be marked. 

M -a long-term bandwidth threshold related to 
allowable "burst" size. 

BYTES - a number of bytes in a packet being 
received, from a customer's terminal, by an access 
node. 

COUNT - the value in the accumulator. 
THRESH - a variable threshold. 



Bandwidth Monitoring and Packet Marking - Al- 
gorithm^ 

One of those algorithms, Algorithm (A), is 
shown in FIG. 3 with a graphic example thereof 
illustrated in FIG. 5. 



Initialization for the Algorithm (A) 

1 . Set the accumulator variable COUNT to 0. 

2. Set the threshold variable THRESH to S 



Steps in the Algorithm (A): 

1. During each interval, upon receipt of each 
packet from the customer's terminal (FIG. 3): 
5 a. Set the byte count variable BYTES to 

the number of bytes in the packet 

b. Compare the value of COUNT with the 
value of THRESH and take the following actions: if 
COUNT < THRESH, then pass the packet on un- 
w marked and replace the value of COUNT by 
COUNT+BYTES + k. Otherwise, if COUNT 2 
THRESH, then mark the packet, pass it on. and 
keep the same COUNT. 

2. At the end of every interval (FIG. 4): 
75 a - Replace the value of COUNT by 

COUNT-c or 0, whichever is larger. 

b. Set THRESH to COUNT + S or M 
whichever is smaller. ~* 

In FIG. 5 the vertical axis presents the value in 
20 bytes for the parameter k, the value of BYTES for 
the current packet, the value of the short-term 
threshold S and the value of the long-term* thresh- 
old M, the value of the threshold variable THRESH, 
and the decrement constant c. The horizontal axis 
25 is time-divided by equal intervals I. Interval I is the 
duration between two instants of time, such "as the 
instants tt) and t1. Also along the horizontal axis 
there is a series of numbers, each of which iden- 
tifies a specific arriving packet 
30 Parameter k is shown in FIG. 5 as upward- 

directed arrows with solid arrowheads. It is a con- 
stant value for all packets of a virtual connection 
with the value typically in the range 0-1000. Ac- 
cording to the algorithms, COUNT is incremented 
35 by the parameter k every time a packet arrives. 

The value BYTES is represented in FIG. 5 by 
upwardly directed arrows with open arrowheads 
and various values for the several arriving packets. 
According to the algorithm, COUNT is incremented 
40 by BYTES every time a packet arrives, except 
when the packet is to be marked. 

The accumulated values of COUNT are shown 
in FIG. 5 as heavy horizontal line segments. 

The values of THRESH are shown in FIG. 5 as 
45 a light dotted line. 

When the packet arrives, COUNT is compared 
with THRESH. If COUNT is less than THRESH, the 
packet is not marked and COUNT is incremented. 
If COUNT is equal to or greater than THRESH, the 
so packet is marked and COUNT is not incremented. 

Stars are positioned above the lines 
(representing the parameter k and BYTES) of pack- 
ets which are to be marked because those packets 
are determined to be transmitted at an excessive 
55 transmission rate. 

The decrement constant c is shown in FIG. 5 
by downwardly directed open headed arrows, 
which are of constant value and occur at the end of 
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each interval I except when COUNT would be 
decremented below zero. 

Also in the algorithm of FIG. 4, the sum of 
COUNT plus the value of S is compared with the 
value of M. If the sum is greater than M, THRESH 
is set to M. If the sum is less than or equal to M. 
THRESH is set to the sum of COUNT plus the 
value of S. 

Once the decision is made to mark the packet 
or not and the packet header is marked appro- 
priately, the packet proceeds through the access 
node 20 of FIG. 1 to an output controller before 
being put into an output buffer associated with the 
output link, through which the packet is to be 
transmitted. At that time, the information in the 
packet header field, reserved for the marking sig- 
nal, is forwarded to a packet dropping logic circuit 
53, associated with node 30 of FIG. 1. A more 
detailed block diagram of the packet dropping logic 
circuit 53 is presented in FIG. 6. 

In FIGS. 6 and 7 there is shown both a block 
diagram and the algorithm for the logic of deter- 
mining whether or not to drop the current packet 
which is about to be applied to the output buffer of 
the access node 30 or which is about to be applied 
to the output buffer of any of the switch nodes 40, 
50, 60 and 70 of FIG. 1 . 

It is assumed that traffic is light to moderate in 
the nodes 30, 40 and 60 along the virtual connec- 
tion. At the switch node 50, however, traffic is 
heavy enough to create a congestion condition. 

First of all, the packet dropping logic circuits 
53 and 54, which are associated with lightly loaded 
nodes 30 and 40 and are like the logic circuit 55 of 
FIG. 6, run the algorithm of FIG. 7. Since there is 
no congestion at these nodes and the output buff- 
ers are not full when tested, whether or not the 
current packet is marked, it is passed to the output 
buffer of the relevant node for transmission along 
the virtual connection. 

Next the packet traverses the node 50, the 
congested node. The packet is applied to a drop- 
ping circuit 90 in the packet dropping logic circuit 
55 by way of a lead 91. Before the packet is 
placed in its identified packet output buffer 92 by 
way of a lead 93, that buffer is checked to deter- 
mine whether or not it is full. If the buffer 92 is full, 
a signal is forwarded through a lead 94 to the 
dropping circuit 90. Regardless of whether or not 
the packet is marked, if the output buffer 92 is full, 
the packet is dropped. Whether or not the output 
buffer is full, a measurement of congestion is de- 
termined. The number of packets in the packet 
output buffer 92 is applied by way of a lead 95 to a 
congestion measuring circuit 96. At the same time, 
a signal representing the availability of processor 
real time is applied to the congestion measuring 
circuit 96 by way of a lead 97, 



In response to the signals on the leads 95 and 
97. the circuit 96 produces a signal on a lead 98 
indicating the amount of congestion that exists at 
the node 50 of FIG. 1. The congestion signal on 
5 the lead 98 and a threshold value applied by way 
of a lead 99 determine whether or not marked 
packets are to be dropped by the dropping circuit 
90. A signal indicating that a packet should be 
dropped is transmitted by way of a lead 100 to the 
10 switch node 50. When the packet dropping signal 
occurs, the relevant packet is dropped (if it is a 
marked packet) or is placed in the output buffer 92 
for subsequent transmission through the link 76 of 
FIG. 1 (if it is an unmarked packet). The aforemen- 
15 tioned procedure drops marked packets before 
placing them in the output buffer. It is called "input 
dropping" because packet dropping is done at the 
input to the buffer. Alternatively, one may place all 
the packets in the output buffer if there is space 
20 there and implement "output dropping" for marked 
packets. That is. when a marked packet in the 
output buffer finally moves up to the head of the 
queue and is ready for output, the threshold is 
checked, and the packet will be dropped or trans- 
25 mitted accordingly. 

The congestion measure is a threshold picked 
to guarantee that a certain quantity of resources 
are available in the node for passing unmarked 
packets. A weighted sum of the number of packets 
30 ' residing in the output buffer 92 plus the amount of 
real time available in the processor is used to 
measure congestion. The amount of real time is 
related to the parameter k. When the parameter k 
is equal to zero, real time is not a concern. Then 
35 the number of packets in the output buffer is the 
only measure of congestion. 

Since the packet is dropped at the node 50 
when that node is congested and the packet is a 
marked packet, the congestion is somewhat re- 
40 lieved. The packet being dropped is one previously 
marked at the access node 30 as a packet being 
transmitted at an excessive transmission rate. A 
congestion condition, less critical than a full output 
buffer, is relieved by dropping only the marked 
45 packets. Under such a condition, unmarked pack- 
ets are passed through to their destination. 

Congestion, therefore, is relieved for the most 
part by dropping the packets of customers who are 
transmitting at rates which exceed their agreed 
so upon, or assigned, transmission rate. The network, 
therefore, can adequately serve all subscribers who 
transmit within their subscribed transmission rate. 
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Alternative Bandwidth Monitoring and Packet Mark- 
ing ■ Algorithm (B) 



Turning now to FIGS. 8 and 9. there is shown 
an alternative algorithm for determining which 
packets .are to be marked together with a graphical 
presentation of how that algorithm operates on a 
series of received packets. Only the first step (the 
marking step) is different from the earlier described 
algorithm explained with respect to FIGS. 3 and 4. 
Initialization and parameters have the same or a 
similar meaning except that a new value of COUNT 
is determined before the value of COUNT is com- 
pared with the value of THRESH. The value of 
BYTES plus the parameter k is added to the value 
of COUNT before the comparison occurs. Subse- 
quently when the value of COUNT is determined to 
be less than the value of THRESH, the packet is 
not marked and is passed along. The existing value 
of COUNT is retained. If COUNT is equal to or 
greater than THRESH, the packet is marked and 
passed along. Then the value of COUNT is de- 
cremented by the sum of BYTES plus the param- 
eter k. As with the earlier described packet marking 
algorithm. Algorithm (A), packets which are marked 
are considered to be only those being transmitted 
at an excessive transmission rate. 



Initialization for the Algorithm (B): 

1 . Set the counter variable COUNT to 0. 

2. Set the threshold variable THRESH to S. 



Steps in the Algorithm (B): 

1 . During each interval upon receipt of each 
packet from the customer's terminal (FIG. 8): 

a. Set the byte count variable BYTES to 
the number of bytes in the packet 

b. Replace the value of COUNT by 
COUNT + BYTES + k. 

c. Compare the value of COUNT with the 
value of THRESH and take the following actions: If 
COUNT < THRESH, then pass the packet on un- 
marked. Otherwise, if COUNT £ THRESH, then 
mark the packet, pass it on, and replace the value 
of COUNT by the value of COUNT-BYTES-k. 

2. At the end of every interval I (FIG. 4): 

a. Replace the value of COUNT by 
COUNT-c or 0, whichever is larger. 

b. Set THRESH to COUNT + S or M, 
whichever is smaller. 

FIG. 9 shows the result of the series of packets 
being monitored by the algorithm of FIG. 8. Stars 
positioned above the lines, representing the param- 



eter k and BYTES for the packets, indicate that 
those packets are to be marked as excessive trans- 
mission rate packets. 



Special Service Packet Marking 

A special new service can be offered to cus- 
tomers. The new service offering is based on the 
io fact that the network is arranged for dropping 
marked packets wherever and whenever the 
marked packets encounter a congestion condition 
or a full buffer condition. This new service is a low- 
cost, or economic rate, service. 
is By subscribing to, or choosing, the economic 

rate service, the customer assumes a risk that 
transmitted messages will not be delivered to their 
destination if they encounter congestion. The op- 
erating equipment (either at the customer's termi- 
20 nal or at the access node) is arranged to mark 
every packet transmitted by the economic rate 
service customer. Thereafter, as the marked pack- 
ets proceed through the packet switching network, 
they are treated like other marked packets. If these 
25 marked economic rate service packets traverse a 
point of congestion or arrive at a full buffer in the 
network, the packets are dropped. Since all of the 
economic rate packets are marked, there is a high 
probability that the message will not be delivered 
30 during busy traffic conditions. 

During hours of light traffic, however, there is a 
high probability that there is no congestion nor full 
buffers. At such times, the message is likely to 
traverse the network successfully on the first try. 
35 This economic rate service will benefit both the 

operating communications company and the cus- 
tomer. The customer benefits from the low rate 
charges. The operating company benefits because 
customers of this service will tend to transmit eco- 
40 nomic rate service messages during slow traffic 
times when much of the company's equipment is 
idle and much spare capacity is available. 



45 Congestion Control 

The approach adopted in this invention offers 
the following advantages over existing techniques 
for controlling congestion in a packet switching 

so system. First, it affords customers both a guar- 
anteed level of information throughput as well as 
the opportunity to avail themselves of excess ca- 
pacity that is likely to exist in the network 
(depending on the instantaneous level of network 

55 congestion) to realize information throughputs be- 
yond their guaranteed levels. This provides cus- 
tomers flexibility in trading off factors such as guar- 
anteed and expected levels of throughput, integrity 
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of information transport through the network, and 
the associated costs paid to the network provider 
for different grades of service. 

Second, the approach affords the network pro- 
vider greater flexibility in the provisioning of key 
network resources to meet the demands of cus- 
tomers, in that the network resources can be pro- 
visioned to meet the total demands due to the 
guaranteed average throughputs of ail customers 
(with a high level of certainty) rather than a 
statistically-predicted peak total demand of all cus- 
tomers. This lessens the conservatism required in 
network resource provisioning and allows a higher 
average level of network resource utilization. 

Finally, the congestion control, as illustrated in 
Figures 1-9. is a distributed control that employs 
the monitoring and marking of packets at access 
nodes and the dropping of marked packets at any 
network node that may be experiencing congestion. 
The control is completely decoupled from actions 
that may be adopted by customers at the end 
.terminal equipments. Distribution of the control 
through the network eliminates the need for very 
low-delay signaling mechanisms between network 
nodes that would otherwise be needed if excessive 
rate packets were to be dropped at access nodes. 
The decoupling of the control from terminal actions 
eliminates the dependence of the integrity of the 
control scheme on terminal actions, as is the case 
with some other control schemes. The distribution 
of control provides a robustness to the control and 
an ensured protection of both the network and well- 
behaved customers from other customers who abu- 
sively send packets at excessive transmission 
rates. 

The foregoing describes both apparatus and 
methods for marking packets being transmitted at 
an excessive rate when received at an access 
node, or being transmitted from a special cus- 
tomer, and for dropping marked packets at any 
node in the network when a congestion condition 
exists. The apparatus and methods described here- 
in together with other apparatus and methods 
made obvious in view thereof are considered to be 
within the scope of the appended claims. 



Claims 

1 . A method for dropping a marked data packet 
to be transmitted from a switch node in a packet 
switching network, the method CHARACTERISED 
BY the steps of: 

a. preparing to transmit the data packet; 

b. determining whether or not the data pack- 
et is marked; 

c. evaluating congestion at the output of the 
switch node; 



d. determining whether or not the congestion 
is at a predetermined value; and 

e. if the data packet is marked and the 
congestion is at the predetermined value, dropping 

s the data packet before it is transmitted. 

2. A method for dropping a marked data pack- 
et, in accordance with claim 1. and CHARAC- 
TERISED BY the further steps of: 

f. if the data packet is unmarked, passing it 
io to the output; 

g. multiplexing the unmarked data packet 
with other data packets; and 

h. transmitting the multiplexed unmarked 
data packet and other data packets through a link 

is or terminal line. 

3. A method for dropping a marked data pack- 
et, to be transmitted from a switch node in a packet 
switching network, the method CHARACTERISED 
BY the steps of: 

2 <> a. segregating data packets transmitted by 

one customer; 

b. marking that one customer's data packets 
being transmitted; 

c. preparing to transmit one of the custom- 
25 er's data packets; 

d. determining whether or not the one data 
packet is marked; 

e. evaluating congestion at the output of the 
switch node; 

so f. determining whether or not congestion at 

the node is at a predetermined value; and 

g. if the data packet is marked and the 
congestion is at the predetermined value, dropping 
the data packet. 

as 4. A method for dropping a marked packet to 

be transmitted at an excessive rate from a switch 
node in a packet switching network, and method 
CHARACTERISED BY the steps of; 

a. preparing to transmit the data packet; 

40 b. determining whether or not the data pack- 

et is marked as a data packet being transmitted at 
an excessive rate; 

c. evaluating congestion at the output of the 
switch node; 

45 d. determining whether or not the congestion 

is at a predetermined value; and 

e. if the data packet is marked and the 
congestion is at the predetermined value, dropping 
the data packet. 

so 5. A packet switching node with a plurality of 

received terminals; CHARACTERISED BY a plural- 
ity of channels, each interconnected with a different 
one of the received terminals, for transmitting se- 
quentially packets of data, each packet containing 

55 at least one marking bit; means, responsive to a 
measurement of the congestion of the packet 
switching node, for generating a signal indicating 
the amount of congestion in the packet switching 
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node; and means, responsive to the generated 
congestion signal and a threshold value, for delet- 
ing any packet of data containing a marking bit. 
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